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Abstract 

We consider a noisy Slepian-Wolf problem where two correlated sources are separately encoded 
(using codes of fixed rate) and transmitted over two independent binary memory less symmetric channels. 
The capacity of each channel is characterized by a single parameter which is not known at the transmitter. 
The goal is to design systems that retain near-optimal performance without channel knowledge at the 
transmitter. 

It was conjectured that it may be hard to design codes that perform well for symmetric channel 
conditions. In this work, we present a provable capacity-achieving sequence of LDGM ensembles for 
the erasure Slepian-Wolf problem with symmetric channel conditions. We also introduce a staggered 
structure which enables codes optimized for single user channels to perform well for symmetric channel 
conditions. 

We provide a generic framework for analyzing the performance of joint iterative decoding, using 
density evolution. Using differential evolution, we design punctured systematic LDPC codes to max- 
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imize the region of achievable channel conditions. The resulting codes are then staggered to further 
increase the region of achievable parameters. The main contribution of this paper is to demonstrate that 
properly designed irregular LDPC codes can perform well simultaneously over a wide range of channel 
parameters. 

Index Terms 

LDPC codes, LDGM codes, density evolution, correlated sources, non-systematic encoders, joint 
decoding, differential evolution, area theorem. 

I. Introduction 

Wireless sensor networks have become very popular in recent years and are being increasingly 
used in many commercial applications. A good survey of the problems involved with designing 
sensor networks can be found in [I], P]. A sensor network typically has several transceivers 
(also called nodes), each of which has one or several sensors. The task of these sensor nodes 
is to collect measurements, encode them, and transmit them to some data collection points. 
The topology of sensor networks varies widely with the application, but typically the data from 
all the nodes is transmitted to a central node, also known as a gateway node, before further 
processing is done on the data. This problem is often referred to as the sensor reachback problem. 
There are many constraints on the size and cost of the networks, so the nodes have limited 
computational capabilities, communication bandwidth etc. Hence the nodes have to perform 
distributed encoding, despite having to transmit correlated data. One of the main goals in the 
area of wireless sensor networks is to reduce the amount of transmitted data by taking advantage 
of the correlation between the sources. In many cases, there is generally a medium access control 
(MAC) protocol in place, which eliminates interference between the different nodes. In this case, 
one can assume that each node transmits through an independent channel, from the same channel 
family. A simple sensor network consisting of two sensors is shown in Fig. 1. This problem 
of distributed encoding and transmission over independent channels gives a noisy version of 
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the celebrated Slepian-Wolf (SW) problem. The SW problem was introduced and solved in the 
landmark paper [ ], and shows that the optimal coding scheme suffers no loss in performance (in 
terms of rate) even in the absence of communication between the various encoders. A variety 
of coding schemes have been designed that can achieve the SW bound when channel state 
information is known at the transmitter. 



A. Prior Work 

The first practical SW coding scheme was introduced by Wyner and is based on linear 
error-correcting codes [ ]. Chen et al. related the SW (distributed source coding) problem to 
channel coding via an equivalent channel describing the source correlation [5], [6]. Using this 
observation they used density evolution to design LDPC coset codes that approach the SW 
bound. Distributed source coding using syndromes (DISCUS) also provides a practical method 
to transmit information for this problem when the encoding rates are restricted to the comer 
points of the rate region [7]. 

For transmision over noisy channels, separation between source and channel coding is known 
to be optimal when the channel state is known at the transmitter [8]. When the channel state is 
unknown, it is still desirable to take a joint source-channel coding (JSCC) approach (via direct 
channel coding and joint decoding at the receiver). The main reason is that separate source 
and channel coding requires compression of the sources to their joint entropy prior to channel 
encoding. After that, the variation in one channel's parameter cannot be offset by variation in the 
other channel. Further advantages of JSCC, over separated source coding and channel coding, 
are discussed further in [9]. 

The performance of concatenated LDGM codes has been studied in [ ' and that of Turbo 
codes in [9]. Serially concatenated LDPC and convolutional codes were also considered in [ ], 
where the outer LDPC code is used for distributed source coding. 
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It was conjectured in [12] that LDPC codes do not perform well for the noisy SW problem' 
and that it is hard to design codes that perform well for symmetric channel conditions. In this 
work, we show a sequence of LDGM codes which approach the SW bound for symmetric 
channel conditions. 

B. Universality 

Another interesting line of research in the area of sensor networks is the sensor location 
problem. The sensor locations are optimized in order to collect the most relevant data. A 
possibility of using moving sensors is present in a variety of applications, including air pollution 
estimation, traffic surveillance etc. [2]. A natural consequence of this is the variation in channel 
conditions as a result of sensor mobility. As a result, it may be unreasonable to assume that 
transmitters have detailed channel state information. This problem of unknown channel state at 
the transmitter naturally arises in the context of many multi-user scenarios, including cellular 
telephony. 

For fixed user code rates, reliable communication is theoretically possible over a wide range 
of channel conditions [ 3]. We call a system universal if it provides good performance for 
all system parameters that do not violate theoretical limits. This designation neglects the fact 
that the receiver is assumed to have channel state information and is based on the standard 
assumption that the receiver can estimate the channel state with negligible pilot overhead. While 
irregular LDPC codes can be optimized to approach capacity for any particular channel condition, 
the performance can deteriorate markedly as the channel conditions change. So, we design 
LDPC codes which are robust to variation in channel conditions-. Such schemes are desirable 
because they minimize the outage probability for quasi-static channels (e.g., when a probability 
distribution is assigned to the set of possible channel parameters). 

'The authors consider only systematic LDPC codes 

^Unfortunately, the LDGM codes that achieve the symmetric channel condition are not universal. 
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Fig. 1. System Model 

II. Problem Setup 

Consider the problem of transmitting the outputs of two discrete memoryless correlated 
sources, (f/i,f/2), to a central receiver through two independent discrete memoryless channels 
with capacities Ci and C2, respectively. The system model is shown in Figure 1. We will assume 
that the channels belong to the same channel family, and that each channel can be parametrized 
by a single parameter a (e.g., the erasure probability for erasure channels). The two encoders 
are not allowed to communicate. Hence they must use independent encoding functions, which 
map k input symbols (Ui and U2) to rii and ^2 output symbols (Xi and X2), respectively. The 
rates of the encoders are given hy Ri = k/rii and R2 = k/n2. The decoder receives (Yi, Y2) 
and makes an estimate of (Ui,U2). 

The problem we consider is to design a graph-based code, for which a joint iterative decoder 
can successfully decode over a large set of channel parameters. For simplicity, we assume 
that both the encoders use identical codes of rate R (i.e., R = k/n,ni = 77,2 = n). Reliable 
transmission over a channel pair (01,02) is possible as long as the SW conditions (1) are 
satisfied. 
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For a given pair of encoding functions of rate R and a joint decoding algorithm, a pair of 
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Fig. 2. The SW region for erasure channels, for a fixed rate pair {R, R) 

channel parameters (01,02) is achievable if the encoder/decoder combination can achieve an 
arbitrarily low probability of error for limiting block-lengths (i.e., k — > 00). We define the 
achievable channel parameter region (ACPR) as the set of all channel parameters which are 
achievable. Note that the ACPR is the set of all channel parameters for which successful recovery 
of the sources is possible for a fixed encoding rate pair (R,R). We also define the SW region 
as the set of all channel parameters (01,02) for which (1) is satisfied. The SW region for the 
erasure channel family is shown in Figure 2. 

In this paper, we consider the following scenarios: 



1) The channels are erasure channels and the source correlation is modeled through erasures. 

2) The channels are additive white Gaussian noise (AWGN) channels and the source corre- 
lation is modeled through a virtual correlation channel analogous to a binary symmetric 
channel (BSC). 



These models might appear restrictive, but we believe they provide sufficient insight for the 
design of codes that perform well for arbitrary correlated sources and channels. Our analysis in 
Section III admits general correlation models and memory less channels. 
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A. Erasure Correlation 

The erasure system model is based on communication over binary erasure channels (BECs) 
and the source correlation is also modeled through erasures. Let Z he a Bernoulli -p random 
variable and X,X' be i.i.d. Bernoulli-^ random variables. The sources Ui and U2 are defined 



We have H(Ui\U2) = H{U2\Ui) = 1 — p and H{Ui, U2) = 2 —p. This correlation model can be 
incorporated into the Tanner graph (see Section III-A, III-C) at the decoder with the presence 
or absence of a check node between the source bits depending on the auxiliary random variable 
Z. Note that the decoder requires the realization of the random variable Z, for each source bit, 
as side information. Because of this requirement, one might consider this a toy model that is 
used mainly to gain a better understanding of the problem. Still, a very similar model was used 
recently to model internet file streaming from multiple sources [ ]. 

This model can also be thought of as having two types of BSC correlation between the source 
bits (as described in the next section), one with parameter and one with parameter 1. The 
correlation parameter p determines how many bits are correlated with parameter 1. The receiver 
knows which bits are correlated with parameter 1. 

B. BSC Correlation 

A more realistic model is the BSC/AWGN system model, where communication takes place 
over a binary-input additive white Gaussian-noise channel (BAWGNC) and the symmetric source 
correlation is defined in terms of a single parameter, namely p = Pr(t/i = U2). It is useful to 
visualize this correlation by the presence of an auxiliary binary symmetric channel (BSC) with 
parameter 1 — p between the sources. In other words, U2 is the output of a BSC with input Ui 
i.e., U2 = Ui + Z. Here Z is a Bemoulli-(l — p) random variable and can be thought of as an 
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error. Let /i2(-) denote the binary entropy function. Then, H{Ui\U2) = H{U2\Ui) = /i2(p) and 
H{UuU2) = l + h2{p). 

This correlation model can be incorporated into the Tanner graph at the decoder (described 
in Section III-C) as check nodes between the source bits, with a hidden node representing the 
auxiliary random variable Z (which carries a constant log-likelihood ratio log attached to 
the check node. For this scenario, the decoder does not require any side information i.e., it does 
not need to know the realization of the auxiliary random variable Z. 

C. Existence of Universal codes 

In this section, we discuss the existence of universal coding schemes, for the system model 
considered in Figure 1. Let /^^(Xi; Yi) and (-^2; Y'2) denote the mutual information between 
the channel inputs and outputs when the channel parameters are given by «i and «2- The 
following theorem shows the existence of codes which have large ACPRs. 

Theorem 1. Consider encoders with rate pair {R, R). For a fixed pair of channel conditions 
{ai,a2), which are not known at the transmitter, random coding with typical-set decoding at 
the receiver can achieve an average probability of error Pe,ai,a2 bounded above by 2^"'^("i'"2)^ 
where 

7(ai,a2) = min{/,,(Xi;yi)-i?i7(?7i | U2), 
Ia2{X2;Y2) - RH{U2 I U^), 
J,,(Xi; Fi) + I^,{X2; K,) - RH{Ui, U2)}. 

Hence, there exists an encoder for which the probability of error 

P < 9^"7("li"2) 

Proof: This follows from extending the proofs in [ ] to the SW problem. ■ 
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Remark 1. A simple application of Fano's inequality shows that any pair of channel parameters 
for which 7(0:1, 02) < are not achievable (the probability of error is strictly bounded away 
from zero). For binary memoryless symmetric (BMS) channels, the condition 7(0:1, 02) > 
translates to the conditions in (1). So, the conditions in (1) are both necessary and sufficient for 
transmission over BMS channels. 

Remark 2. For BMS channels, the achievable channel parameter region for a random code is a 
dense subset of the entire SW region for limiting block-lengths. This follows by using Theorem 1 
and applying the Markov inequality. This result is also easily extended to random linear codes. 

We conclude that, for a given rate pair (R,R), a single encoder/decoder pair suffices to 
communicate the sources over all pairs of BMS channels in the SW region. Thus, one can 
obtain optimal performance even without knowledge of (01,02) at the transmitter. We refer to 
such encoder/decoder pairs as being universal. This means that random codes with typical- set 
decoding are universal for BMS channels. 

While random codes with typical-set decoding are universally good, encoding and decoding is 
known to be impractical due to its large complexity. This motivates the search for low complexity 
encoding/decoding schemes which are universal. 



III. Analysis 

A. LDGM Codes 

Assume that the sequences Ui and U2 are encoded using LDGM codes with a degree 
distribution pair (A, p). Based on standard notation [ ], we let \{x) = AjO;*"^ be the degree 
distribution (from an edge perspective) corresponding to the variable nodes and p{x) = J2i Pi^^~^ 
be the degree distribution (from an edge perspective) of the parity-check nodes in the decoding 
graph. The coefficient Aj (resp. pi) gives the fraction of edges that connect to the variable nodes 
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Fig. 3. Tanner Graph of an LDGM (LT) Code with erasure correlation between the sources 

(resp. parity-check nodes) of degree i. Likewise, Lj (resp. Ri) is the fraction of variable nodes 
(resp. check nodes) with degree i. 

Since the encoded variable nodes are are attached to the check nodes randomly, the degree of 
each variable node is a Poisson random variable whose mean is given by the average number 
of edges attached to each check node. This mean is given by m = -R'(l), where -R'(l) is the 
average check degree. Therefore, the resulting degree distribution is L{x) = e"^*^^"^). Throughout 
this section, we consider the erasure correlation model described in Section II-A. 



The Tanner graph [ ] for the code is shown in Fig. 3. Code 1 corresponds to the bottom half 
of the graph, code 2 corresponds to the top half and both the codes are connected by correlation 
nodes at the source variable nodes. One can verify that the computation graph for decoding a 
particular bit is asymptotically tree-like, for a fixed number of iterations as the blocklength tends 
to infinity. This enables the use of density evolution to compute the performance of the joint 
iterative decoder. 



Let X£ and ye denote the average erasure probability of the variable nodes at iteration i for 
users 1 and 2 respectively. The density evolution equations [ ] in terms of the variable-node 
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to check-node messages can be written as 

x^+i = [(1 -p)+pL {g{e2, ye))] A (^(ei, xe)) 

ye+i = [il-p)+pL{g{ei,xe))]X{g{e2,ye)), 

where g(e, x) = 1 — (1— e)p(l— x). Notice that, for LT codes, the variable-node degree distribution 
from the edge perspective is given by X^'^^x) = L^^\x) because \{x) = L'{x)/L'(l) = L{x), 
when L(x) is Poisson. With this simplification, the density evolution for symmetric channel 
conditions (ei = €2 = e) can be written as 

xe+i =[{l-p)+pX{l-{l- e)p(l - xe))]\ (l - (1 - e)p(l - x,)) . (2) 

This recursion can be solved analytically, resulting in the unique non-negative p{x) which satisfies 

X = [(1 -p) +pA(l - (1 - e)p(l - x))] A(l - (1 - e)p(l - x)). 

The solution is given by 



-1 / y/{l-p)^ + Ap{l -x)-{l-p) 

= ~n V ■ Tr- 
ail -e) y 2p 

00 sr^i-l (2i-l\ k 



1 \ ^ l^k= 

[l-e]^ til 



-x\ 



which is not a valid degree distribution because it has infinite mean. To overcome this, we define 
a truncated version of the check degree distribution via 

I ^fc^ol k )P i I AT 

N P Z^i=l j(l+p)2i-l X ^X 



P Kx) 



GN{p)=Y.HfH. 



\2i-\ ' 

/, I I —I— ( / 



for some p > and G N. This is a well defined degree distribution as all the coefficients 
are non-negative and p^(l) = 1. The parameter p increases the number of degree one generator 
nodes and is introduced in order to overcome the stability problem at the beginning of the 
decoding process [17]. 
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Theorem 2. Consider transmission over erasure channels with parameters ei = €2 = e. For 
G N and /i > 0, define 



G 



N 



N spi-l (2i-l\ k 

l^k=o V k_Jf_ 

2j-l ' 



+ P) 



and m 



/i + gjv(p) + l 
1-e 



r/zen, m f/ze limit of infinite blocklengths, the ensemble LDGM(^n, \{x), {x)^ where 

\[x) = e'"^"-^) and p^{x 
enables transmission at a rate R 



(4) 



/i + Gjv(p) + l 
(i-'=)(i-g — ) ^ ^^Y/z a ^/f error probability not exceeding 1/N. 



lJL+l-p/2 

Proof: See Appendix A. ■ 
From Theorem 2, we conclude that the optimized ensemble LDGM(n, A(x), p^(a;)) can 
achieve the extremal symmetric point of the capacity region. Unfortunately, one can show that 
this ensemble cannot simultaneously achieve both the extremal symmetric point and the comer 
points of the SW region. In Figure 4, this can also be observed numerically via the density 
evolution ACPR (DE-ACPR) of this ensemble for = 2048. 



B. Puncturing and LDPC Codes 

In [18], it is shown that correlated codes are suboptimal when transmitting correlated sources 
over independent channels. The conditions in (1) implicitly assume the use of uncorrelated codes 
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Fig. 4. ACPR (Density Evolution tiiresiiold) of tiie optimized (erasure channel) LT Code with A'^ = 2048 
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Fig. 5. Tanner Graph of an LDPC Code with source correlation 

i.e., we require the average mutual information (over the code ensemble) I{Xi; X2) = 0. 

This condition is clearly not satisfied when we use a systematic LDPC ensemble. This also 
explains the loss in performance of systematic LDPC codes when compared to Turbo codes, as 
shown in [ ]. To ensure the independence of the transmitted symbols, we use LDPC ensembles 
with punctured systematic encoders. 



C. Density Evolution for LDPC codes 

Assume that the sequences Ui and U2 are encoded using LDPC codes with a degree distri- 
bution pair (A, p) and a punctured systematic encoder. Let the fraction of punctured (systematic) 
bits be 7. 

The Tanner graph [\f] for the joint decoder is shown in Figure 5. Codes 1 and 2 correspond 
to the bottom and top half of the graph. The codes are connected by correlation nodes attached 
to the punctured bits. The joint iterative decoder proceeds in rounds, by alternating one round 
of decoding for code 1 with one round of decoding for code 2. Let and hi denote the density 
of the messages emanating from the variable nodes at iteration i, corresponding to codes 1 and 



^Assuming that the transmission alphabet is {±1}, the densities are conditioned on the transmission of a +1. 
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2. The density evolution equations [16] can be written as follows 

dBMSC 



■>i+i 



(5) 

7/(L(p(a^))) + (1 -7)bBMsc ® A(p(b£)), 



where A(a) = A,a®(*-i), L(a) = Ua^'^^-'K p(a) = E.P^a^^^^'^ ^bmsc and bgMsc are the 
densities of the log-likelihood ratios received from the channel. The function / at the correlation 
nodes depends on the equivalent channel corresponding to the correlation model, as described 
in [5]. Although one cannot assume that the all-zero codeword is sent simultaneously by both 
users, one can show that this DE recursion suffices for typical message pairs. 

First consider the BSC correlation model. By symmetry of the problem, we can assume that 
user 1 transmits the all-zero codeword and the second user transmits a typical codeword. Due 
to the constraints imposed by the correlation, the fraction of ones in the systematic part of the 
codeword is 1—p. Density evolution proceeds with two types of messages (those connected to a 
variable node with transmitted value +1 and those connected to a variable node with transmitted 
value —1). By symmetry of the message passing rules [16, p. 210], we can factor out the sign for 
the messages connected to variable nodes with transmitted value —1. This sign can be factored 
into the correlation node (once again by the symmetry condition). The fraction of correlation 
nodes which are flipped is 1—p. So, we introduce a parity-check at the correlation nodes which 
evaluates to a BemouUi-p random variable i.e., /(a) = aBsc(p) S a- This simplification enables 
us to proceed with density evolution assuming the transmission of an all-zero codeword for both 
the users. 

Note that such a simplification is not necessary for the erasure correlation model. For a BEC 
correlation with probability p, there is a parity-check at the correlation node with probability p 
and with probability 1—p there is no parity-check, so /(a) = (1 — + pa. 

The residual error probability at iteration i, (6^,62), is computed using the error functional 
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(B{-) defined in [ , p. 201]: 

e{ = e(^ -if(L (p(b£))) + (1 - 7)aBMsc ® L{p{ae)) 



-ff(L (p(a^)) ) + (1 - 7)bBMsc ® L{pibe)) 



D. Staggered Block Codes 

It is well known that single-user codes perform well at the corner points of the SW region. 
Although single-user codes do not perform well for symmetric channel conditions, they can 
be used to construct staggered codes that perform well at the corner points and for symmetric 
channel conditions. Consider 2 sources with Lk + (1 — (3)k bits each. Without loss of generality, 
add (3k zeros at the beginning for source Ui and add (3k zeros at the end for source U2, to 
get {L + l)k bits. We call (3 the staggering fraction. Next encode each block of k bits using a 
punctured {n — k, k) LDPC code. The rate loss incurred by the addition of (3k zeros can be made 
arbitrarily small by increasing the number of blocks L. At the decoder, one has the following 
structure: The performance of this staggered structure can be understood by considering the 
erasure case in the limit L — )• 00. 

Theorem 3. Consider transmission over erasure channels with erasure rates (€1,62) using ca- 
pacity approaching punctured {n — k, k) LDPC codes. The staggered block code (with staggering 
fraction (3 ) allows reliable communication for channel parameters 

ei < min{l - - 1 - R{1 - and 

62 <l-i?(l-p(l -/?)), 

where R = k/{n — k) is the design rate of the code. 

Proof: Consider the first block for source Ui. The parity bits see a BEC(ei) channel and 
the source bits see an effective BEC(1 — (3) channel (assuming no information comes from the 
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Fig. 6. Decoder structure for staggered codes 

decoder on the other side). So the effective erasure rate at the first block is (1 — R')ei + R'{1 — 13) 
(R' = k/n is the rate of the code before puncturing). The code can decode as long as R' < 
1 - ((1 - R')ei + R'{1 - (3)) i.e., ei < 1 - - (3). Suppose the first block of Ui can 
decode successfully, then the source bits in the first block of f/2 see an effective channel of 
(1 — /3){1 — p) + j3. The parity bits see a channel with erasure probability £2. So, the effective 
channel seen by the first block of the second code is (1 — R')e2 + R'{1 — p(l — (3)). So this block 
can be decoded as long as 62 < 1 — R{1 — p{l — /?)). The decoding continues by alternating 
between blocks of Ui and U2. This proves the claim. ■ 



Corollary 1. Consider transmission over erasure channels using capacity approaching punctured 
{n — k,k) LDPC codes. The staggered block code (with staggering fraction (3 = 1/2) allows 
reliable communication at both the corner points and the symmetric channel condition. 
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Proof: The proof follows by matching the conditions of the previous theorem to a corner 
point and the extremal symmetric point of the SW region. ■ 
For general channels we can analyze the performance of the staggered code using density 
evolution. Let i G and a^*^ and b^*"* denote the density of the messages emanating 

from the variable nodes at iteration corresponding to codes 1 and 2 in block i. The DE 
equations can be written as follows: 

^^"^ - [l {Pf{L (p(bf^)))) + (1 - (p(b?)))) + (1 - 7)aBMsc] ® A(p(a,)) 



(6) 

7 ((1 - P)f{L (p(a«))) + (p(af'^)))) + (1 - 7)bBMsc] ® A(p(b,)). 



Here, a^*^ bf* = A+oo (the delta function at oo) for i ^ {1, . . . , L}. 

E. Differential Evolution 

Throughout this section, we use x to denote an element of M" for some n E N, and Xi to 
denote its ith component. Let V = {i\Xi 7^ 0} and P = {z | pi 7^ 0} be the support sets of 
the variable and parity-check degree distributions respectively, which are assumed to be known. 
The correlation parameter p is fixed. We design LDPC codes for this scenario using differential 
evolution [ ], for a design rate R^. Let 



A"-i = IxeW 



n 

Xi = 1, Xi > 0,i = 1, ■ ■ ■ ,n 

1=1 



denote the unit simplex and Uy = |V|, Up = \V\. Then, the search space for all variable (check) 
degree profiles is A""^^ (A"^^^). The optimization is performed over the search space S = 
^n^-i y. /\np~i^ ^jj-j^ parameter vectors x = [xx,XpY, where xx G A""^-'^,Xp G A"p"^. In 
our optimization procedure, we expand the search space to S' = {x E W""'^"'p,'^-{xx)i = 
1, J2ii^p)i — 1}' simplicity in the crossover stage. We generate an initial population of trial 
degree distributions by uniformly sampling the degree distributions from the unit simplex. 



'*{xx, V) and (xp, P) correspond to the variable and parity node degree profiles respectively. 
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Let C be a finite subset of cliannel parameters (ai,a2) tliat correspond to tlie sum rate 
constraint of the SW conditions for a design rate Rd- Let F : 5' x C — t- [0, 1] x [0, 1], 
(x,ai,a2) ^ (^1,62) be the function that gives the residual error probability^ (using joint 
density evolution as described in Section III-C) for each decoder, for a pair of codes with degree 
distribution x (i.e., {xx,Xp)), when transmitted over channels with parameters (ai,a2). We use 
discretized density evolution [i l]^ to compute the performance of an ensemble. 

For our design, we want the code to achieve an arbitrarily low probability of error on C and 
we want the rate of the code R{x) to be as close to the design rate Rd as possible. So, we define 
the cost function. 



if X E S and J^{x) = 00, if a; G S'\S. The constants a and b are chosen through trial and 
error. The parameters chosen for the designs considered in this paper are r = 10^^, a = 10 and 
b = 30. The optimization is then setup as min^g^/ J^(x). 

We use a variant of differential evolution, with the mutation and recombination scheme given 
in [20]. The resulting codes are then staggered as described in Section IH-D. 



This paper shows that the SW conditions are necessary and sufficient for communication of 
correlated sources through independent BMS channels, without channel state information at the 
transmitter. This implies that a single random code is sufficient to communicate with vanishing 
probability of error, for the entire SW region. We showed the achievability of the symmetric 

^We set the maximum number of iterations to 100 for all the designs considered in this paper. Density evolution is stopped 
when the maximum number of iterations is reached or the difference in the residual error probability between successive iterations 
is less than 10~*. 

*A 9 bit linear quantization is used over a likelihood ratio range [—20, 20] 




IV. Results and Concluding Remarks 
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channel condition under message passing by providing a sequence of LDGM ensembles which 
can achieve an arbitrarly low probability of error. 

We designed punctured systematic LDPC codes for the scenarios described in Section 11. 
The design was performed to maximize the ACPR, in contrast to previous work. For the erasure 
correlation model, the optimization was performed for a design rate of Rd = 0.57 after puncturing 
and source correlation p = 0.5. The resulting degree profile 



A(x) = 0.3633X + 0.2834x2 + 0.2315a;^ + 0.1217x^^ 
p{x) = 0.531776a;^ + 0.468224a;^ 



has a design rate of 0.3308 and transmission rate 0.4962. The ACPR for this code is shown 
in Figure 7 along with the SW region for the rate pair (0.4962,0.4962). This shows optimized 
ensembles can achieve a large portion of the SW region. 

The BSC source correlation parameter was p = 0.9 and the optimization was performed for 

1 

0.9 
0.8 
0.7 
0.6 
£"0.5 
0.4 
0.3 
0.2 
0.1 

"o 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 

Fig. 7. ACPR (Density Evolution threshold) of an optimized (erasure channel) LDPC Code of rate 0.3308 is shown in blue. 
The grey area is the ACPR after staggering. 
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Fig. 8. ACPR (Density Evolution threshold) of an optimized (AWGN channel) LDPC Code of rate 0.323 is shown in blue. 
The grey area is the ACPR after staggering. 

a design rate i?^ = 0.5 after puncturing. The resulting degree profile 

A(a:) = 0.26725a; + 0.26823x2 + 0.07557x^ + 0.212a;^ + 0.027898x^ + 0.0061593x^+ 

0.0011654x^^ + 0.14173a;^^ 
p(x) = 0.37856a;^ + 0.56211x^ + 0.0080803x^ + 0.028448x^^ + 0.0095319x^^ + 0.013267x2^ 

has a design rate of 0.323 and transmission rate 0.476. The ACPR for this code is shown in 
Figure 8 along with the SW region for the rate pair (0.476,0.476). These results show that 
ensembles optimized using differential evolution almost achieve the entire SW region. 



Appendix A 
Proof of Theorem 2 

We will use the following Lemma to show that the density evolution equations converge to 
zero at the extremal symmetric point. 

Lemma 1. 

^ > /V^i ' forO<x<l--. 
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Proof: For 0<x<l — we have 



/i 



/i + Giv(p) + 1 



/U + G7v(p) + 1 /i + G'7v(p) + l 

+ p(x) 
/x + G'^(p) + l' 



(7) follows from the fact that 



z(l+p)2-i ^ AT + l ^ N+1 1-x 

i=N+l ^ i=N+l i=N+l 



The last step follows from explicit calculations, taking into account that 0<x<l — ■ 
From (2), the convergence criteria for the density evolution equation is given by 

x> [(l-p)+pA^(e,x)] A^(e,x), 
where X^{e,x) = A (l - (1 - e)p^(l - a;)). We have, 

A^(e,x)=e-'^(i-^>''"(i-^) 

<e ""^^ ^i^+Gjv(p)+i [fx>— (8) 
' - 

^ _ v/(l - py + Apx - (1 - p) 



a/(1 — p)^ + 4px — (1 — p) 

where (8) follows from Lemma 1. The polynomial f{y) = py'^ + (1 — p)y — x is a convex 
function of y, with the only positive root at y = V(i-rt^+4px-(i-p) _ ^ ^ ^(i-,)2+4p.-(i-rt ^ 

then f(y) < 0. Hence, [(1 — p) +pX{e,x)~\ A(e,x) — x < and the density evolution equation 
converges, as long as x > So, the probability of erasure is upper bounded by 
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Note that p'^^^x) dx is a monotonically increasing sequence, upper bounded by 1 — |. So, 

in the limit of infinite blocklengths the design rate is given by 

R = lim ^0 ^(^^^^ = {1 - e){l - e-'^) 
p(^){x)dx 
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